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As implementation of the Missouri Comprehensive Guidance Programs continues at 
the local district level, the task, of evaluation emerges. To achieve accountability for 
guidance programs at the local district level in Missouri, three types of systematic 
evaluation are currently underway. The three types are Program Evaluation, Personnel 
Supervision and Evaluation, and Results Evaluation. 



Program Evaluation 

Question: Is the local district's comprehensive guidance 

program in place and functioning as it was designed 
to function? 

Fvaluation 

Procedure: Missouri School Improvement Program Standards 
and Indicators pertaining to guidance and counseling 
(see Appendix A) 

Personnel Supervision and Evaluation 

Question: Are district school counselors and other guidance 
personnel being supervised and evaluated based on 
job descriptions derived directly from the district 
comprehensive guidance program? 

Evaluation 

Procedure: Missouri Performance-Based Evaluation for School 
Counselors — Formative and Summative ■ 
(see Appendix B) 

Result" Evaluation 

Question: What impact does the local district's comprehensive 
guidance program have on its students, teachers, 
parents, the school, and the community? 

Evaluation 
Procedures: 

1 . Guidance Curriculum 

a. Develop an instrument (Missouri Guidance 
Competency Evaluation Survey) to measure student 
confidence in guidance curriculum competency 
attainment (see Appendix C for the 6-9 Survey) 

b . Use the Missouri Guidance Competency 
Evaluation Survey to identify the factor structure 
(subscales) of the guidance curriculum scope and 
sequence 4-6, 6-9, 9-12 (see Appendix D) 
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c . Establish the reliability and validity of the Missouri 
Guidance Competency Evaluation Survey 

(see Appendix E) 

d. Field test the Missouri Guidance 

Competency Evaluation Survey to measure pre-post 
gain scores on guidance competencies taught in 
guidance curriculum unit K-12 (see Appendix F) 

2. Individual Planning 

a. What — Student Career Folders/portfolios/four year 



plans 



b. 



How 



Review and judge content and use by 
students, procedures to be developed 



3. 



Responsive Services 



a. 



What 



Individual counseling, small group 
counseling, consultation, referral 



b. 



How 



Procedures to be developed 



4. 



System Support 



a. 



School climate - goals 
(see Appendix G) 



b. 



User/patron satisfication — use surveys 
(see Appendix H) 
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Appendix A 

MISSOURI SCHOOL IMPROVEMENT STANDARDS 



RESOURCE STANDARDS 

Resources alone do not represent a quality educational program. Without them, however, 
educational programs of desirable quality are unlikely. Resources provide the buildings in which 
schooling talces place, the professional and support staff necessary to operate schools, and all the 
insu-uctional materials and apparatus necessary to implement a curriculum. Although resources 
must be variable to address the diverse needs of the students and the community, the resource 
standards describe the basic programs, staffing patterns and resources that should be present in 
every Missouri school district. 



3.2 Guidance and Counseling Staff - The guidance and counseling program in each school 
attendance center is implemented by a certificated counselor and sufficient other stall to 
ensure that the program is effectively implemented. Counselors are employed and 
assigned consistent with the following student/staff ratios: ••■ 



MINIMUM 




DESIRABLE 


STANDARD 




STANDARD 


tudents 


m 


Student* 


£TE 


1-100 


.20 


1-75 


.20 


101-200 


.40 


76-150 


.40 


201-300 


.60 


151-225 


.60 


301-400 


.80 


226-300 


.80 


401-500 


1.00 


301-375 


1.00 


501-600 


1.20 


376-450 


1.20 


601-700 


1.40 


451-525 


1.40 


701-800 


1.60 


526-600 


1.60 


801-900 


1.80 


601-675 


1.80 


901-1000 


2.00, etc. 


676-750 


2.00, etc. 



PROCESS STANDARDS 



The Missouri School Improvement Standards (primed in bold type) will be applied to each school 
district in the state. School districts will use these. standards to evaluate their educational processes 
and to make plans for district-wide improvements. Each standard may be met in a variety of ways 
at the local district level Some of these ways qr6 described by the indicators (or descriptors) 
which follow each standard. Districts may also meet these standards through the use of indicators 
other than those described. In the assessment process, a team of educators will validate the elf oris 
a district has made to achieve the objectives which are embodied in the standards. This evaluation 
will concern the relevancy of the district's efforts and achievements to the stated standards. 

Some indicators have been identified as starred (*) indicators. These indicators should always be 
addressed in the written review report and will be utilized in determining the degree to which the 
district is meeting the intent of the standard. Relevant information in relation to non-black-dot 
indicators should' also be included in the repoit. 
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GUIDANCE AND COUNSELING 



11.1 A A comprehensive guidance and counseling program is implemented in 
each attendance center and is considered an integral part of each 
school's total educational program. 

r 

* 1 . The written guidance program contains a definition of the program and a statement of 

philosophy and objectives. 

* 2. Guidance personnel and teachers work together in addressing the guidance and placement 

needs of students. 

3. An advisory committee has been established and maintained that provides support, offers 
advice and reviews program activities. 

4. There is a professional person at the district and building level who directs the overall 
administration and coordination of the guidance program. 

11.1 B Structured, developmental experiences are presented systematically 
through both classroom and group activities in grades K-12. 

* 1 . A needs assessment has been canned out which identifies the student competencies to be 

addressed through the curriculum component, as well as other components of the 
comprehensive guidance program. 

* 2. The guidance curriculum adequately addresses idendfied student needs/competencies in 

career planning/exploradon, knowledge of self and others, and educational and vocational 
development, 

* 3. The guidance curriculum and program is articulated through grade levels to ensure 

continuity of learning, extension of skills, knowledge and competencies. 

11.1 C Assessment, advisement and placement activities are available at 

appropriate levels to ail students to assist them to manage their personal, 
educational and career development decisions. 

* 1 . Individual student planning activities are provided to help students assess and interpret data 

and information related to their abilities, interests, skills and achievement. 

* 2. Appropriate and effecdve activities are used to provide all students with job-seeking and 

job-retention skills. 

3. Appropriate recruitment, orientation, selection criteria and assessments are used for 
admission into the area vocational school and for placement into occupadonal training 
programs. 

4. Appropriate job development and placement acuvides are conducted. 
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111 D students ing ' consultation and referral activities are provided for 

* 1 . Consultation is provided to teachers, administration and parents relative to individual 
student s educational progress and personal and social concerns. """viauai 

2. Counseling is provided on a small group and individual basis. 

3 . Guidance personnel use other professional resources of the school and communkv io refer 
students when appropriate. • lo IUU 

11.1 E Management activities that support the total school program and 

maintain, enhance and evaluate the total guidance program are in place 
Guidance personnel have access to resources of the tvpe and in the 
quantity needed to implement the guidance and counseling program. 

1 . The guidance facilities, equipment and resource materials are appropriate and cordate \vi th 
tne goals ot the program. 

"2. A procedure has been developed to assess the decree to which the program has be-n 

implemented K- 12, the degree to which program objectives have been met and the impact 
the program is haying on students and the school. 

3 . Guidance personnel participate in inservice programs and professional development 
activities to keep them up-to-date on professional practices, coinmunitv resources and labor 
market information. 

4. Systematic follow-up studies are conducted to determine the decree to which the district 
educauonai program is meeting the needs of students and compiles with federal and state 
regulations. 

5 . A guidance budget has been developed with input from the counselors and stah that reflects 
the guidance needs in the district. 
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Appendix B 



Counselor Performance-based Evaluation 
Formative and Summative Evaluation Forms 

During the 1984-1985 school year, Missouri's statewide committee on performance based 
evaluation developed criteria to be used in the evaluation of counselors. These criteria were 
intended as part of the performance based teacher evaluation process. The specific items 
identified were initially organized by a statewide committee of school counselors. It was 
revised in 1989 using the Missouri Comprehensive Program Model Structure. 

Because the evaluation criteria for school counselors relate directly to the Comprehensive 
Guidance Program, and the job descriptions of school counselors based on the Program, it 
is essential for evaluators to confer with counselors early in the school year regarding these 
criteria. 



Formative Evaluation 

Three forms or instruments provide the basis for gathering information and setting goals 
during the formative evaluation phase. These are the Pre-Observation Worksheet, the 
Formative Observation Form and the Job Target Sheet. 

The Pre-Observation Worksheet is completed prior to the pre-observation conference and 
discussed with the evaluator during the pre-observation conference. Use of this worksheet 
provides the evaluator with an understanding of goals and activities prior to the scheduled 
observation. It is important to note that many criteria relate to the establishment of direction 
and goals for the counseling guidance program. It is suggested that evaluators confer with 
counselors early in the year to discuss the goals established for this program. This may be 
accomplished during the pre-observation conference prior to a scheduled observation. 

During the observation, the evaluator takes sequential notes, recording specifically the 
activities, events anr, relevant statements observed. From these notes the evaluator 
completes the Formative Observation Form. This instrument allows the evaluator to record 
pertinent information about performance criteria in a useful and logical manner. The 
Formative Observation Form is completed following both scheduled and unscheduled 
observations. 

The Job Target Sheet assists in the attainment of identified goals. To complete the Job 
Target, the performance area and the specific criterion statement for the desired objective 
should be identified. Then the desired improvement objective should be stated in terms 
similar to the descriptor(s) for that criterion. The procedures for achieving the objective 
should include the activities and responsibilities of both the counselor or librarian and 
evaluator. The appraisal method and target dates describe the means by which the evaluator 
will determine if the objective has been accomplished and the schedule for that 
accomplishment. 
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Summative Evaluation Report for Counselors 



The Summative Evaluation Report provides a means of synthesizing information obtained 
during the Formative Evaluation phase. It lists the major performance areas and the 
specific criteria for each area. Each criterion statement is rated according to the 
performance level demonstrated by the counselor during the formative phase. The 
appropriate performance level descriptor should be circled by the evaluator to clearly 
identify the rating for that criterion and focus attention on that performance level. 

The Performance Expectation column represents the level of performance expected of 
effective counselors. This is the level toward which all should strive. The next column 
represents exemplary performance and would be typical of only a small percentage of the 
ratings given. This column should be used with prudence, so that the frequency with 
which it is used does not dilute the significance of the Performance Expectation column. 

A rating below Performance Expectation should have been preceded by efforts to improve 
that performance through the use of Job Targets. A rating below Performance Evaluation 
means the counselor is not meeting that criterion at an acceptable level, and appropriate 
plans for improvement should be continued. 
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Missouri Guidance Competency Evaluation 

Survey - Grades 6 - 9 

Developed by 

Norman C. Gysbers, Richard T. Laparu 
Karen D. iMulton, and Leslie Eastman Lukin 



STUDENT NAME (Last, First, Middle Initial) 
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GENDER 



O Male 
O Female 



GRADE 



06 08 
Q7 Q9 



ETHNIC HERITAGE 



O Hispanic 

O Black 

O White 

O American Indian or Alaskan Native 

© Asian or Pacific Islander 
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AREA I CAREER PLANNING AND EXPLORATION 



Piease rate how confident you are that you could successfully perform the following career planning and : 
• exploration tasks. Blacken the circle that indicates the level of your confidence for each item. ■ 

LEVEL OF CONFIDENCE 

! AM CONFIDENT: 

1. that i understand my interests and abilities and how they help me make a career choice. _© © 

2 that I know how to handle adult disapproval if I have an interest in choosing a class 
usually taken or a job usually filled by the opposite sex. 



3. that I know how to find out which leisure activities are best for me. 

4. that I know how to explore careers in which I may be interested. _ 



5. that I understand the importance of making plans for the future Gobs, vocational, 
technical education, employment and training programs, college, and military). _ 



6. that I know about leisure activities I can do when I am older.. 



7. that all classes and jobs are acceptable for both females and males 

° that I know about different hobbies, sports, and activities in which I could get involved, 
y. that I understand how to prepare for careers in which I may be interested 



0. that I can handle "kidding" from other students if I have an interest in choosing a class 
usually taken, or a job usually filled by the opposite sex. 
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AREA II KNOWLEDGE OF SELF AND OTHERS 



j Please rate how confident you are that you could successfully perform the following tasks related to under- j 
1 standing yourself and others. Blacken the circle that indicat es the level of your confidence for each item, j 

LEVEL OF CONFIDENCE 

I AM CONFIDENT: 



I. that I know how to get along with my parents, step-parents or guardians. . 
hat I know how to accept responsibility for my decisions. 
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i that I can effectively communicate my feelings and thoughts. ^ ^ 
i. that I can get along wtr. co:n cevs ar.n nuts - - 
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AREA II KNOWLEDGE OF SELF AND OTHERS 

(continued) LEVEL OF CONFIDENCE 

' \tA CONFIDENT: 

6. that I know what drugs and alcohol do to my body and mind 



7. that I respect other people even though they may be different from me.. 

8. in my understanding of what is important to me 



9. that I know how to help a friend who has a problem with drugs and alcohol.- 



1 1 . that I know how to make good decisions. 

1 2. in myself 



1 3. that I know what to do when I feel down. 



14. that I know how to handle my problems or where to get help. 



15. that I understand how my thoughts and feelings affect my behavior. - 



1 6. that I know how to get along with my brother, sister, step-brother, or step-sister. . 

1 7. that I understand my friends and classmates 



that I know how to identify the real problem when I'm worried or upset.. 

19. that I accept and like myself, both physically and mentally 

20. in stating my own ideas : 



21 . that I know how to evaluate my decisions and change bad ones... 



22. that I know how f he use of drugs or alcohol may affect my relationship with others. 

23. that I know what is important to me so I can set goals 



24. that I know how to come up with many possible solutions to a problem. 

25. that I know how to help with family responsibilities 



26. that I know some things that cause problems in families (moving, divorce, or 
unemployment) 



27. that I know how to be a good fnend.. 



28. that I can talk to sompone when I need to.. 



29. that I know where to get help for myself or a friend who has a problem with drugs or 
alcohol 



'10. that I know how to find help when my family has problems.. 
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AREA III EDUCATIONAL AND VOCATIONAL DEVELOPMENT 



•Please rate how confident you are that you could successfully perform the following educational and! 
vocational development tasks. Blacken the circle that indicates the level of your confidence for each item.: 



LEVEL OF CONFIDENCE 

o 
x 



1 AM CONFIDENT: 


Very Low 
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o 
■c 
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r Low nor 


c» 
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CO 
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E 
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c 
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1 . that I know what employers expect nf workers. 
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2. that I know how academic skills such as math, reading, and science relate to my 
career goals 
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3. that I know how to look for a part-time summer joh 
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4. that I know what johs are availahle (orally 
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5. that I understand the challenges sturients have in high schnnl 
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6. that I know how to organize my class anri homework materials 
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/Ok 
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7. that I know how to select high school courses that help meet my needs, interests, 
and career goals 
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8. that I can take gnori notes 
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9. that I understand how completing high school will better prepare me to continue my 
education (vocational technical education, military, anr! college) 
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1 0. that 1 understand high school credits and graduation requirements. 
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1 1 . that I know how to do well on tests 
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12. that I know how to select middle school extra-curricular activities that will meet my 
interests and future goals 
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13. that I understand how completing high school will better prepare me for the job 
market. 


© 


© 


© 


© 


© 


© 


® 


14. that I know the classes offered by my area vocational technical school. 
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15. that I know how to organize my time 
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16. that I know how to listen and ask questions in class. 
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17. that 1 know what will be expected of me in high school. 
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1ft. that I know joh interview skills 
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19. that I know where to go to get help when I have a problem concerning high 

school. 
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20. that I know how to apply for a job. 
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o 1 









21 . that I know how to start and finish my assignments well. Z Z Z I 
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Missouri Comprehensive Guidance Evaluation 
Internal Reliabilities 



Scale 



High School 



rl 


.93 


rl 


.91 




.84 


F4 


.82 


F5 


.85 


F6 


.86 


F7 


.80 


F8 


.82 


F9 


.76 


rlU 


.75 


rl 1 


.69 




.68 


r 13 


.68 


r 14 


.78 


rl j 


O 1 

.81 


F16 


.45 




Junior High School 


Scale 


Alpha 


171 

r i 


.91 




or 

.85 




.85 


r4 


.80 


rj 


. /y 


r;£ 
ro 


HA 


r / 


.57 


F8 


.70 


F9 


.58 




Elementary 


Scale 


Alpha 


Fl 


.82 


F2 


.74 


F3 


.74 


F4 


.78 


F5 


.69 


F6 


.61 


F7 


.62 


F8 


.51 
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Appendix E 



Correlation Between the Missouri Comprehensive Guidance Evaluation Survey (MCGES: Form for Grades 9-12) and 
Cri teria Indices (n=334) 



Criteria Indices MCGES 



Demographic Va riable 

1. Sex # 02 

2. Race # 02 

3. Grade .20* 

Social Desirabi lity Scal e (Hays, Hoyashi. & Stuart, 1989) .00 

Goal Instability Scale (Robbins & Patton, 1985) -.46* 

Superio rity Scale (Robbins & Patton 1985) .28* 

Hope Scale (Snyder et al . , 1991) 

1. Agency (self efficacy) .57* 

2. Pathways (confidence in surmounting obstacle to goals) .52* 

Positive and Ne gative Affect Schedule (Watson, Clark, & Tellegen, 1988) 

1. Positive Affectivity .51* 

2. Negative Affectivity -.21* 

Career Decision Profile (Jones, x988) 

1. Decidedness (regarding career) .32* 

2. Comfort (regarding career decision status) .46* 

3. Self-Clarity .30* 

4. Knowledge About Occupations .29* 

5. Decisiveness .32* 

6. Career Importance .31* 



Correlations Between the Missouri Comprehensive Guidance 
Evaluation Survey and Other Measures of Self-Regard 
(Grades 4-6, N=430; Grades 6-9, N=510) 



Correlations of Areas and Categories with Related Instruments 

Area I: Career Planning and Exploration 

Grades 4-6 Grades 6-9 

Scholastic Compete; ice r».43, p-.OOO r».31, p=.000 

Social Acceptance r=.31, p=.000 r~.26, p-.OOO 

Job Competence r=.26, p=.000 

Social Desirability Scale r=.15, p-.002 r=..2Q, p=.000 
Athletic Competence r=.30 f p=000 

Global Self-Worth r».38, p-.OOO 

•Category A: Exploring (and Planning) for Careers 

Global Self-Worth r«.29, p«.000 r=.37, p=.000 

Job Competence r=.25, p=.000 

Scholastic Competence r=.43, p=.000 r=.31 , p=.000 

♦Category C: Understanding How Being Male or Female Relates to School and Work 

Physical Appearance r=.15, p=.001 

Romantic Appeal r-.12, p=.034 

♦Category N: Learning How to Use Leisure Time 

Athletic Competence r-.31, p=.000 r=.28, p=.000 

Area II: Knowledge of Self and Others 

Scholastic Competence r~.37 ? p=*.000 r=.33, p=.000 

Social Acceptance r=.33, p».000 r=.30 f p=.000 

Global Self-Worth r=.37 ? p=.000 r=.53, p^.OOO 

Close Friendship r«.33, p=.000 

Soc. Desirability Scale r=.29, p=.000 r=.28, p=.000 

Behavioral Conduct r«.33, p^.000 r=.44, p=.000 

•Category B: Understanding and Accepting Myself 

Behavioral Conduct r«.30, p=.000 r«.37, p=.000 

Global Self-Worth r«.34, p-.OOO r=.58, ^.000 

Social Acceptance r=«.28, p».000 r=.33, p^.OOO 
Scholastic Competence r*.33, p=.000 

Physical Appearance r«.46, p-.OOO 

Romantic Appeal r=.32, p=.000 

•Category F: MakL 3 Decisions 

Global Self-Worth r-.30, p«.000 r«.47, p=.000 

Scholastic Competence r«.34, p^.OOO r«.31, p-.OOO 

Physical Appearance r».33, p~.000 

Behavioral Conduct r-.39, p=.000 

•Category H: Understanding and Getting Along With Others 

Social Acceptance r=.33, p=.000 r=.33, p=.000 

Behavioral Conduct r=.29 f p=.000 r=.32 f p=.000 

Close Friendship r«.45 f p=.000 

Global Self-Worth r-.38, p-.OOO 
Scholastic Competence r=.32 f p-.OOO 



(Area II continued on next page) 
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Area II; Knowledge of Se lf and Others (continued) 

Grades 4-6 Grades 6-9 

•Category K: Knowing How Drugs and Alcohol Affect Me and My Friends 

Behavioral Conduct r=.14, p=.003 r«.34, p=.000 

Close Friendship r«.19, p-.001 

Global Self-Worth r~.18, p=.000 r=.32, p«.000 

Social Acceptance r*.20, p«.000 r=».14, p=.002 
•Category P: Learning About Family Responsibilities 

Behavioral Conduct r=.27, p«.000 r».44, p=.000 

Global Self-Worth r=.45, p=.000 

Area III: Educational & Vocational Development 

Scholastic Competence r=.44, p=*.000 r=.36, p=.000 

Global Self-Worth r=.33, p«.000 r=.42, p=.000 

Job Competence r~.31, p=.000 

Social Desirability Scale r*..24, p=».000 r=.19, p=.000 
Behavioral Conduct r=.32, p=:.000 

•Category J: Improving Study and Learning Skills 

Scholastic Competence r*.46, p=.000 r=.46, p=.000 
Behavioral Conduct r=.35, p-.OOO 
Social Acceptance r=.31 , p=.000 

Global Self-Worth r=.45, p=.000 
•Category M: Learning From Friends and Others About School 

Close Friendship r=.34, p=.000 
Social Acceptance r~.33, p=.000 

Global Self-Worth r=.35, p-.OOO 
•Category E: Preparation for Finding Jobs 

Job Competence r=.34, p=.000 
•Category 0: Planning For School 

Scholastic Competence r*.39, p=.000 r=.29, p«.000 



List of Other Measures 

Self-Perception Profile for Children subscales 

- Scholastic Competence - Physical Appearance 

- Social Acceptance - Behavioral Conduct 

- Athletic Competence - Global Self-Worth 

Self-Perception Profile for Adolescents (additional subscales) 

- Job Competence 

- Romantic Appeal 

- Close Friendship 

Social Desirability Scale 
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Appendix F 

Evaluating a Guidance and Language Arts Unit 

for High School Juniors 

Richard T. Lapan, Norman Gysbers, Ken Hughey, and Thomas J. Ami 



In this study the authors report on the evaluation of a program enhancing both the career 
development and writing skills of high school juniors. Participants were J66 juniors attending a 
suburban midwestern high school. School counselors and English teachers worked together to 
provide an opportunity for students to develop academic skills while exploring relevant career 
issues. Results indicated that for a wide range of students, participation in the program led to the 
achievement of specific guidance competencies. Changes in perceived mastery of guidance compe- 
tencies predicted positive change in vocational identity and attainment of higher English grades for 
girls. Implications for promoting school counselors' involvement in guidance program evaluation 
practices are discussed. 



With the implementation of comprehensive school guidance 
programs well underway in many stales and school districts 
across the country, developing evaluation strategies for these 
programs is now a priority. Gysbers, Hughey, Starr, and Lapan (1992) 
argued that such evaluation practices should answer five basic 
questions: 

1 . Are the program's contents, structure, and resources in place in 
the school? 

2. Arc counselors supervised and evaluated based on their job 
descriptions? 

3. Are procedures used to measure students* mastery of guidance 
competencies? 

4. Are procedures used to measure the impact of the program on 
the climate and goals of the school? 

5. Are the individuals served by the program and the community 
satisfied *vith the program? 

This study connects mastery of guidance competencies to academic 
and career development outcomes. Evaluating results of such programs 
can provide school counselors with vital feedback to refine current 
practices to more effectively meet student needs. 

Several assumptions of the Missouri Comprehensive Guidance Pro- 
gram (MCGP) establish guidelines for programs school counselors 
develop and evaluation strategies used to improve these programs. First, 
the MCGP attempts to reach all students. Second, a strong emphasis is 
placed on having certified school counselors present structured guid- 
ance activities in the classroom. Third, these activities are targeted to 
help students attain specific developmental competencies identified by 
the state's Department of Elementary and Secondary Education. Fourth, 
because Missouri defines guidance as an integral part of each school's 
total educational program, the guidance curriculum component of the 
program addresses both social -emotional developmental concerns and 
academic achievement through MCGP program content areas: knowl- 
edge of self and others, careerplanning and exploration, and educational 
and vocational development. In this article the authors report findings 
from an evaluation study of a joint guidance and language arts program 
developed to meet these guidelines. 

Since 1985, school counselors and English faculty members at a 

444 



midwestern public high school have co-sponsored a guidance learning 
unit designed to help all juniors achieve specific guidance competencies 
and language arts skills. Ami and Bancroft (1989) described a cooper- 
ative spirit between English and guidance faculty members, which aided 
in the creation and implementation of a guidance learning unit integrat- 
ing developmental guidance competencies into the language arts curric- 
ulum. Counselors were concerned that all juniors achieve those compe- 
tencies in the MCGP career planning and exploration and educational 
and vocational development content areas, which help students plan fcr 
postsecondary educational -^d vocational transitions. 

The local school district places strong emphasis on composition an£ 
research skills. More than 80% of the students pursue a college educa- 
tion, with most of the remaining students entering postsecondary voca- 
tional education programs. Bancroft (1990) described the challenge the 
English department faced identifying researchable topics that were 
relevant and of interest to all juniors. The English faculty agreed tha 
guidance competencies addressing educational and vocational transi- 
tion issues provided a guidance content area within which writing anc 
research skills could be effectively taught. Guidance and English facul- 
ties worked together to 9 .ace this guidance learning unit within the 
academic schedule for the language arts curriculum presented to all 
juniors. Among other activities, counselors administered and inter- 
preted a vocational interest inventory and an aptitude test. Students used 
the extensive career planning and exploration information available it. 
the Guidance Resource Center. English teachers presented instructior 
on how to write research papers. Students worked with counselors tu 
explore potential postsecondary educational and vocational transitions 
The results of this exploration became the subject matter for a research 
paper required by tneir English teacher. The program is described \v 
more detail in the Method section of this article. 

To help the guidance and English faculty begin to assess the impar 
of their program, an assessment strategy containing three interrelate 
questions was proposed. First, it was important to assess the extent r 
which students met competencies targeted by the program. Secon< 
factors predicting who was more likely to attain these competenci* 
were assessed. Knowledge of these factors might aid in adjusting ar 
tailoring presentation of the program to heterogeneous clusters of st* 
dents, fhird, given an understanding of who achieved these guidan. 
competencies, we assessed whether such attainment predicted posit; 

Journal or Counseling & Development • March/April 1993 • Volume 



EMC 



30 



Evaluating a Guidance and Language Arts Unit for High School Juniors 



zc in vocational identity (Holland, Daiger, & Power, 1980) and 

.Tiic grades assigned for the unit by the students' English teacher. 

o answer these questions, LISREL path analytic models (Joreskog 
orbom, 1989) were developed separately for boys and girls. Ey 

loping separate models for boys and girls, a more insightful explc- 
♦n of Sex by Treatment Interactions was possible. From the materials 

in the guidance unit, three exogenous and nine endogenous vari- 

: were identified and placed into a theoretical model consistent with 
measurement and criterion validity evaluation strategies suggested 
vysbers et ai. (1992). Exogenous variables are those whose predtc- 

lay outside of the focus of this study. They included grade point 
rage (GPA), degree of disinterest in vocational and school subjects, 
: an assessment of a statistical regression phenomenon that could 
taten the internal validity of the study. Endogenous variables are 
se whose prediction was the fecus of the study. They included 
glish achievement, level of academic comfort, extroverted or Lntro- 
tzd personality orientation, four guidance competency category 
as, change in vocational identity, and English grades earned for the 
t. Hypotheses for how the exogenous variables predict the endoge- 
is variables are presented first Following that, hypotheses for how 
ogenous variables predict other endogenous variables are presented. 
First, v:e predicted that GPA would directly influence English grades 
the unit and would indirectly affect attainment of guidance compe- 
tes though its relationship with academic comfort aft3 English 
ievement scores. Second, we were concerned that lower levels of 
stallized vocational interests would negatively affect the attainment 
guidance competencies, change in vocational identity, and English 
' x s. Prior research suggested that flat vocational interest profiles 

. individuals who may lack appropriate experiences with activities 
:vant to developing vocational interests (Prediger & Swaney, 1985) 
\ career maturity (Super, 1983). Thus, we assessed whether such 
ividuals could effectively use a program such as this joint guidance 
i language arts unit. Third, we predicted that students scoring low on 
vocational identity pretest measure would exhibit substantially 
her vocational identity posttast change scores. Variance in vocational 
miry change explained by this path would be due to a statistical 
%-ession artifact. Tracing such effects would help rule out interna! 
.dity threats to the study. Entering such a variable into the model 
r.ved statistical control where design limitations could not be 
;tded. 

The partem of hypothesized relations among the endogenous vari- 
~j are as follows. First, attaining MCGP guidance competencies in 

categories of planning and developing careers, making decisions 
jut college, vocational selection and training, and understanding how 
ng male or female relates tc jobs and careers would predict positive 
vnge in vocational identity and higher English grades. Second, per- 
:ali:y orientation (i.e., extroversion- introversion) would influence 
un.ncnr of guidance competencies and change in vocational identity. 

net English grades. Third, a< .demic comfort and standardized 
giish achievement scores were expected to mediate attainment of 
iancc competencies and English grades, but not vojational identity. 
Evaluating this program in its naturally occurring field setting posed 
-*.rJ problems. Over a period of time, counselors and teachers had 
" v eti ,.ut an understanding of when and where the program fit into 
suage arts curriculum. Because the intervention took place at the 
linnm level, students could not be randomly assigned to treatment 
•ni, mo* conditions. Although these constraints made in analysis v 
.4 x vrr« effects r\ ore :i'Txuit. ;hc\ :ejrt*nted the euicanc \rc ir 
• M ~ ;r.'.f v»'Vr i *<*rvcc ! e .caocrr:* c^cc;:!. """ : 



change due solely to the program was evaluated within this one-group 
pretest-posttest design. 

Cook and Campbell (1979) argued that the one-group pretest-post- 
test design can be cautiously interpreted if certain internal validity 
threats can be ruled out. In this article we demonstrate how structural 
equation modeling can be used to trace the impact of such potential 
confounds to inferring treatment-caused change. Statistical u . Adeline 
procedures enable evaluation efforts to measure treatment impact and 
differential rates of student growth within the workplace restrictions of 
the public school setting. 



METHOD 



Students 



AH juniors attending a suburban midwestern public high school partic- 
ipated in this joint guidance and language arts unit. This sample included 
both higher achieving honors' students and nonhonors* students. In 
accordance with MCGP goals, the program was delivered to all juniors. 
Demographic data on these students are presented in the Results section. 

Guidance and Language Arts Unit 

The unit is designed to include all juniors in the high school. The unit 
took place during 1 3 class periods spread over 8 weeks. The first phase, 



TABLE 1 

Program Outcomes for Honors' and 
Nonhonors 1 Students 



Honors' 



Nonhonors' 



Variable 



M 



t 

SO Value 



M 



SD 



t 

Value 



Vocational 
identity 

Pretest 

Posttest 
English grades 

Boys 

Girls 
Planning and 
developing careers 

Boys 

Girls 
Making decisions 
about college 

Boys 

Girls 

Vocational selection 
and training 
Boys 
Girls 

Understanding how 
being male or 
female relates to 
joes and careers 

8oys 

Gins 



9.61 
11.36 

0.90 
0.94 



6.18 
6.27 



3.73 
3.64 



1.54 

1.36 



2.73 
2.54 



4.97 
5.28 

0.09 
0.06 



1.60 
1.80 



1.01 
1.00 



0.52 
0.49 



0.79 
0.80 



4.09* 



ns 



ns 



ns 



10.34 4.040 

1 1 .74 4.030 

0.61 0.230 

0.92 0.100 



6.18 1.030 
6.04 1.000 



3.67 0.940 
3.61 0.890 



1 .27 0.450 
1.20 0.410 



2.69 0.802 
2.41 0.610 



4.79* 



3.70' 



ns 



ns 



-2. W 



Vae u onor: i - 38. Monhonors n=»*28. 
'r "z j0' ~s~nct significant. 
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days 1-3, involved introducing students to the unit and administering 
an aptitude survey and interest inventory. The second phase, days 4-13. 
involved interpreting the aptitude survey and an interest inventory* 
orienting students to the Guidance Resource Center, orienting students 
and providing them with a tour of the local Vocational-Technical Area 
School, and providing information on prcempioyment training in job 
seeking and job-keeping skills. Individual English classes spent 4 days 
in the Guidance Resource Center using occupational and educational 
reference materials and computen7.ed career information systems to 
explore occupations and post-high-school educational programs related 
to their interests and abilities. For the English department, the following 
skills were emphasized: selecting and using references, organizing 
information for personal use, summarizing information, and using var- 
ious sources of information. Students were graded on their achievement 
of these skills as a regular pan of their work in English. Tnc guidance 
department emphasized the following skills: connecting self-under- 
standing to possible occupational and educational alternatives, explor- 
ing several occupational areas, and developing tentative postsecondary 
plans. The high school counselors* vocational-technical counselor, and 
teachers worked closely together over the duration of the unit. 



lected from student records. GPAs are on a 4-point scale, with an 
equaling 4. 

Missouri Mastery and Achievement Test (MM AT; Missouri Depa 
ment of Elementary and Secondary Education, 1990) scores also w*-. 
collected for each student. The MMAT is 44 a battery of criterio 
referenced achievement tests thai evaluate educational objectives ide 
dried by the Missouri Department of Elementary and Second:. 
Education' 4 {MDESE, 1990, p. 3). The subjects covered by the MM> 
are English and language arts, mathematics, science, and social stud-- 
and civics. Scale scores range trcm 1 to 750. with most students scori 
between 100 and 600. English and language arts achievemer.t sect 
earned by students at the end of their sophomore year were used m t« 
analyses. 

English grades. From teacher grade books. English grades earn* 
for this unit were obtained. Grades were convened to percentiles i 
permit across-class comparisons. Teachers assigned grades on the bas- 
of writing and research work completed by the student. 

Interest measures. The guidance unit included having honors 1 st«. 
dents complete the Strong Interest Inventory (SIX: Hansen & Campbei 



Variables 

Guidance competencies. Student self-report was used to assess achieve- 
ment of specific guidance competencies covert? ^ to-, activity. Students 
were asked to respond yes or no to questions relating achievement of 
guidance competencies due to their participation in the program. Guid- 
ance competencies addressed in the unit were from the MCGP. Com- 
petencies targeted by the program were identified from MCGP career 
exploration and planning and educational and vocational development 
content areas. Students responded to specific competencies from the 
following categories: planning and developing careers (five competen- 
cies); understanding how being male or female relates to jobs and 
careers (two competencies); making decisions about college (three 
competencies); and vocational selection and training (one competency). 
For t tests and path analyses, each student received a single value for 
each competency category. This vaiue represented the total number of 
competencies for each category a student indicated that he or she met 
by participating in the unit. In addition to the achievement of guidance 
competencies, students were asked about their satisfaction with the 
guidance unit and its usefulness to them. 

Vocational identity. The Vocational Identity (VI) scale of the My 
Vocational Situation Scale (Holland, Daiger. & Power, 1980) was used 
as a pre- and posttest measure of vocational identity. According to 
Holland et al, (1980), vocational identity refers to "the possession of a 
clear and stable picture of one* goals, interests, personality, arid 
talents" f.p. I ). The VI scaJc consists of 1 8 true-false items. Scale scores 
ranee from 0 to 18, with higher scores being indicative of a stronger 
sense of vocational identity. Holland ct al. (1980) reported KR-20 
-liabilities at .86 for high school students. The VI scale was adminis- 
tered during the first session of the guidance unit. A second adrninistra- 
5 ion of this instrument was conducted following completion of this unit. 
VI pretest scores were also used to create a dichotomous variable for 
each subject. Students sconng at or above mean VI pretest scores 
received a vaiue of 1. Students scoring below the VI pretest mean 
-ived a vaiue of 0. This variable enabled an estimation of apparent 
•J-i:tf!C :n vocational identity due rolely to where students place in the 
-are distribution. Path models used this variable to measure 
1 :vaiment effects due mainly to a statistical regression artifact, 
w ..; -it achievement. Cumulative GPAs of students were col- 



TABLE 2 

Student Self-Report of AccompHshing Specific 
Guidance Competencies 



Guidance Competencies 



% Boys % Girls 



Area 1 : Career Planning and Exploration 
Category A: Planning and Developing Careers 

1 . l 9'fectively explored possible careers and 
the world ol work. 

2. \ explored several different careers in 
specific areas in which I 3m interested. 

3. My knowledge of how to prepare for my 
career interests has improved. 

4. I have, at least, some tentative plans after 
graduation (job opportunities, training 
programs, college/vocational technical 
schools, financial aids, etc). 

5. I have a better understanding of how 
abilities are related to making career 
choices. 

Category C: Understanding How Being Male or 
Female Relates to Jobs and Careers 



1. II 



2. 



I have an improved understanding of the 
role o( women in today's work force. 
I understand my opportunity to enter 
careers traditionally held by members ol 
the opposite sex. 
Category D: Making Decisions About College 

1. 1 improved my knowledge about various 
colleges and what they offer. 

2. I have gamed insight in how to choose 
colleges that would prepare me for a 
career. 

1 have gained insight as to what careers 
certain majors would preoare me for and 
the future of those careers. 
Area 3: Educational and Vocational Development 
Category O — Vocational Selection and Training 
t. I have a better understanding of my 
vocational interests, aptitudes, and 
abilities. 

Overall Student Satisfaction With Program 
• . THs entire career exploration activity is 
oenefioat and worth continuing. 



62.7 
57.5 
74.7 

96.4 
78.3 

65.1 
63.9 
3C.7 
74.7 



3. 



73.5 

68.7 
39.2 
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70.3 
54.1 
85.1 

97.3 
82.4 

73.0 
79.7 
36.5 
79.4 
70.3 
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-.332 



Low VI Pretest Scores 



Change in VI Scores 



-.121 



.749 



Extroversion/Introversion 
\ 
.459 



Guidance Competencies 



Dislike Response Percentages 



MMAT English 



G.P.A. 





Planning and Developing Careers 
.465 I 

Making Decisions about College 

365 283 | 

Vocational Selection and 
Training 



Understanding Male/Female 
issues 



English Grades 



FIGURE 1 
Structural Equation Model for Boys 



Path Weights represent LISREL Maximum Likelihood estimates. T values for all estimates included in the model are > 2.0. MMAT « Missouri Master/ 
:vement Test for English taken in Spring 1 989. Guidance competencies were reverse scored. Negative weights indicate that greater attainment cf that 
etency is significantly correlated with other variables. Positive weights suggest that tower attainment of that competency is related to other variables. 



) and nonhonors' students take the Career Assessment Inventory 
; Johansson, 1985). Given this situation, comparable measures 
selected from each scale to assess differential treatment impact, 
the SII and the CAI have scales measuring one's preference for 
ing alone with things and ideas* as opposed to working with other 
e. Scores cn these scales were included in path analyses to assess 
feet of extroversion-introversion personality orientation on treat- 
effectiveness. The Academic Comfort scale (SII) and the Educa- 
. Orientation scale (CAI) were chosen to measure the degree of 
Drt one feels in an academic setting. These scores were included in 
ith analyses to assess the extent to which the unit might differen- 

benefU students who are more comfortable and engaged in the 
»chool setting. Bom interest measures report the total percentage 
ms that participants indicate that they dislike. This variable pro- 

a means of assessing treatment effectiveness tor students who 

possibly be less career mature (Super, 1983) or lack appropriate 
iences to promote the crystallization of vocational interests (Pre- 
& S waney, 1 985). We hoped that information obtained from using 

variables in the path analyses would suggest refinements and 
orients that could be made in the guidance and language arts unit 
re effectively meet individual student needs. 



~ai Equation Models 

nalytic models were developed separately for both boys and girls 
LISREL 7 PC software (Joreskog & Sorbom. 1989) 



RESULTS 



Demographics 



Data from 166 high school juniors (90 boys, 76 girls) were analyzed. 
The mean age for boys was 16.24 years (SD=.54). Of the boys. 86% 
were Caucasian, 1 1 .7% African American, and 2. 1 % Asian American. 
With a mean GPA of 2.70 (SD=.67) t 15.7% of the boys were enrolled 
in an honors* English class. The mean age for girls was 16.22 years 
(50=.42). Of the girls, 90% were Caucasian, 6.6% African American, 
and 2.6% Asian American. Having a- mean GPA of 3.07 (SD=.64), 
28.9% of the girls were enrolled in an honors' English class. Girls had 
a higher overall mean GPA in both honors' and nonhonors' groups than 
did boys. 

Program Outcomes 

Table 1 reports data separately for honors' and nonhonors' stude:v.s. 
Significant changes in pre -post VI scores were found for both groups. 
Within the honors' group, boys and girls did equally well on their 
English grades for this project. Within the nonhonors' group, girLs 
received better English grades. Also, girls indicated that from tneir 
participation in this guidance unit, they gained a better understanding 
of how being male or female relates to jobs and careers « guidance 
competencies for career planning and exploration: re a '. ; Oat-jpr C;. 
There were no ^tgnificar.t differences, nowevtr. berwe :; <ji ! or..* 
sn the honors' and nonhonors' samples f 'or the toilo v. juiJar.ce 
competency . lienor.?! planning ±*\*i loom 2 oareer. "a/.:r<g izc- 
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Low VI Pretest Scores 




Change in VI Scores 



Guidance Competencies 
Planning and Developing Careers 
.451 \ 
Making Decisions about College — 

-.260 



English Grades 



FIGURE 2 
Structural Equation Model for Girls 



Tabic 2 summarizes student self-reports of accomplishing specific 
guidance competencies through this developmental program. A total of 
1 1 guidance competencies from two Missouri model content areas (i.e., 
areer planning and exploration and educational and vocational devel- 
opment) were identified to evaluate student reactions. 

Of the 5 competencies from planning and developing careers (Cat- 
egory A), both bovs and girls strongly believed that participation m the 
guidance program helped them to improve their ability to plan and 
explore careers; 96% of the boys and 97% of the girls believed that they 
had at least tentative post-high-school vocational or educational plans 
Only 54% of the girls, however, believed that they had explored several 
different careers in areas of interest. Both boys and girls believed they 
improved their understanding of how being male or female relates to 
jobs and careers (Category C); notably, approximately 80% of the girls 
believed that they had a better understanding of their opportunity to enter 
careers traditionally closed to them. For making decisions about college 
(Category D), more than 75% of both boys and girls believed that this 
guidance unit improved their knowledge of colleges, how to choose a 
college that would prepare them for acareer, and how careers and certam 
college majors are related. In addition, more than two thirds of girls and 
bovs believed that they gained a be tter understanding of their vocational 
interests, aptitudes, and abilities by participating in this program (voca- 
iionai selection and training; Category 0). Overall satisfaction with the 
program was very high. Approximately 95% of the girls and 89% of the 
boys indicated that this guidance unit was beneficial and worth 
continuing. 



competencies, change in VI, and English grades. Boys who met tne 
competencies in the planning and developing careers category had 
higher V] change scores than did boys who scored iower on the guiaanca 
competencies in this category. Boys with higher CPAs earned higher 
English grades. In addition, boys scoring below the pretest mean on the 
VI had greater pre-to-post change scores than did boys scoring above 
the pretest mean on the VI. No significant relationships were found 
connecting attainment of guidance competencies and English grades. 

Several factors mav have contributed to boys' not meeting the 
competencies in the guidance category understanding how being male 
or female relates to jobs or careers. First, boys who did not meet the 
competencies in the planning and developing careers category were no 
likely to have met the competencies in the making decisions abcu 
college and the understanding how being a male or female relates to joo 
and careers categories. Second, boys with higher GPAs and highe 
English achievement scores were less likely to have met these guidanc- 
competencies. Third, boys who had greater numbers of dislikes on th 
SII and the CA1 were identified as being more introverted and were al* 
less likely to gain greater understanding of how being male or femal 
relates to iobs auid careers. 

Bovs who did not meet the planning and developing careers son 
petenc'ies were less likely to accomplish the making aeasior.s aoo: 
college competencies and the vocational selection and training com? 
tencv Level of academic comfort did not predict differential stumme' 
of guidance objectives. Boys with fewer vocational dislikes. !:n:i- 
GPAs. and higher English achievement scores were much more co: 
fortable in an academic setting. 



Structural Equation Model for Boys 



Structural Equation Model for C Vis 



Figure 2 presents 



linear structural equations that Wggesl a marxe. 



Figure I presents linear structural equations that suggest these max.- " ~ m !ikcHhcod ;:umates (or girts. Girls 

mum likelihood estimates that begin to explain mastery of guidance -irfere... ^ n. . 
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TABLE 3 

ndness of Fit Measures for Program Evaluation 
Path Models 



Evaluating a Guidance and Language Arts Unit for High School 



Values 



urea 



juare 

es of freedom 
less of fit index 
:ed goodness of fit index 
nean square residual 
cation indexes >9.0 
ed multiple correlations 
attonaJ identity change 
lish grades 

tning and developing careers 
ing decisions about college 
ationai selection and training 
erstanding how being male/femaie 
lates to jobs and careers 
lemic comfort 
VT English scores 
oversion/lntroversion 



Boy« 


Girls 


21.230 ns 


19.730 ns 


25.000 


29.000 


0.364 


0.958 


0.888 


0.888 


U.U4o 


0.051 


none 


none 


0.130 


0.310 


0.230 


0.200 


0.220 


0.210 


0.140 




0.390 




0.570 


0.480 


0.520 


0.380 


0.56O 


0.420 



MAT = Missouri Mastery Achievement Test, ns « not significant 
sed model fits the data). Blanks for squared multiple correlations 
3 that no significant predictors for this variable were found. 



ed they accomplished the making decisions about cciiege cate- 
irned higher EngJish grades for the unit. For girls, GPA was not 
English grades. Girls who met the competencies in the 
. 0 jnd developing careers category had higher VI change scores 
d girls who scored lower on the competencies in this category. 
' to boys, girls scoring below the pretest mean on the VI had 
pre-to-post chaiig^scores than did girls scoring above the pretest 
n the YL None of the remaining variables listed in Figure 2 
:d attainment of guidance competencies, VI change scores, or 

grades. GPA, MMAT English scores, extroversion-introver- 
d vocational dislikes, however, were related in the same manner 

described earlier for boys. 

aforementioned described structural equation models fit the 
ice matrices for both boys and girls (see Table 3). For boys and 
•wever, relatively low R 2 values for VI change scores, English 
and the competencies in the making decisions about college 
' point to the need to consider additional predictors of these 
jous variables. 

DISCUSSION 

ct al. (1980) contended that the construct vocational identity, 
jred by the VI, focuses on whether or not an individual has 
:d a "clear and stable" (p. 1) understanding of his or her 
al interests, career goals, aptitudes, and personality. The guid- 
does not attempt to push students prematurely toward making 
iecjsions. Rather, students are encouraged to develop a clearer 
iaing of themselves and possible future directions. In this 
'n nonors' and nonhonors ' students reported greater ciariry and 

r participating in (he joint guidance and language arts unit. 

. is boistercd by the fact that mean VI pretest scores cauai 
. :.xcced mean normative scores expected for a high school 
v Holland etai.. 1980) Given m»s starting point. partic:oation 



in the program was clearly accompanied by a substantial increase m 
vocational identity. 

Students enjoyed the guidance and language arts unit and were ab!e 
to discriminate among guidance competencies mastered through parte - 
ipation in the unit. More than 90% of the students said that the guidance 
and language arts unit was beneficial and worth continuing. As report 
in Table 2, most students claimed that they had successfully p*rfcrnec 
a range of activities and met guidance competencies in the categoric: o; 
planning and developing careers, understanding how being male or 
female relates to jobs and careers, making decisions about college, and 
vocational selection and training. Of note, 80% of all girls indicated thai 
because of their participation in the joint guidance and language arts 
unit they had a better understanding of their opportunities to enter 
careers traditionally closed to them. Self-reports of having achieved 
guidance competencies were partially corroborated by the high English 
grades earned by students. 

Path models were developed to identify factors mediating the attain- 
ment of guidance competencies and to establish predictive relationships 
between guidance competencies, change in vocational identity, and 
English grades. Also, such measurement procedures allow exploration 
of potential confounds to inferring treatment-caused change. 

Separate analyses for boys and girls revealed important gender 
differences obscured if only one model had been developed for all 
subjects. For boys, the higher the GPA, the more likely it was that they 
received a higher English grade for the guidance and language arts unit. 
There are three independent ways in which boys did not attain guidance 
competencies in the category understanding how being male or female 
relates to jobs and careers. First, boys who did not meet the competen- 
cies in the planning and developing careers category did not achieve 
competencies in the making decisions about college category, and were 
less likely to report that they have a better understanding of how being 
male or female relates to jobs and careers. Second, boys with higher 
GPA and higher standardized English achievement scores also were less 
likely to meet competencies in these categories. Third, more introverted 
boys who expressed fewer vocational interests were more likely not to 
understand male-female vocational influences after participating in the 
guidance and language arts unit. 

Although both boys and girls earned high English grades, a different 
set of relationships explain this. GPA and standardized English achieve- 
ment scores are unrelated to English grades earned by girls on their 
research papers. The guidance and language arts unit effectively pro- 
moted leaning for girls who were not doing as well in school as it did 
for girls with higher GPA and test scores' In contrast, as stated earlier, 
boys with higher GPA did better than did boys with lower GPA. In 
addition, girls who met the making decisions about college competen- 
cies earned higher English grades for their research papers. The path 
model suggested that girls who did not meet the pianning and develop- 
ing careers competencies were much less likely to meet the making 
decisions about college competencies. These girls were more likely to 
receive lower English grades. It seems that for girls, English grades were 
in part related to the successful completion of the career exploratory 
activities around which the guidance and language arts unit was con- 
structed. To account for this gender difference, we need to look more 
closely at the group of nonhonors* boys. Although equally likely to 
enjoy the unit, achieve guidance competencies, and show change in 
vocational identity, nonnonors' boys received the lowest English grades 
for the unit. 

Mastering guidance competencies in the category planning ind 
developing careen was independent of six background variables ir- 
-ludec in each path analysis. This pattern of relationships was the sa/r.e 
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for both boys and girls. Extroversion-introversion, academic comfort, 
standardized English achievement scores, GPA, level of vocational 
interests, and VI pretest scores did not predict mastery of these compe- 
tencies. Boys and girls who met planning and developing careers 
competencies, however, were much more likely to experience positive 
jiges in vocational identity as a result of the guidance and language 
-*(s unit. 

The unit effectively promoted such learning across sex, levels of 
academic competence, personality orientation, pretest vocational iden- 
tity differences, and crystallization of vocational interests. The joint 
guidance and language arts unit operated independently of these back- 
ground factors to help juniors plan and develop careers. Interestingly, 
for both boys and girls, the guidance and language arts unit promoted 
attainment of planning and developing career competencies irrespective 
of the student's level of vocational identity at the beginning of the unit. 
Students who scored below the VI pretest mean were equally likely to 
meet guidance competencies. Students scoring below the VI pretest 
mean, however, revealed larger vocational identity change scores. Path 
models, forboth boys and girls, identified this as a direct effect operating 
independently of the relationship between the planning and developing 
careers category and vocational identity change. 

There are two plausible interpretations for this finding. First, a 
ceiling effect may increase the opportunities for students scoring below 
the VI pretest mean to exhibit greater change scores relative to those 
scoring at or above the pretest mean. This possibility seems unlikely 
given the test of vocational identity employed. The VI Scale is an 
1 8-item test. Items were not developed to represent any type of difficulty 
hierarchy. The distribution of pretest scores for both honors' and 
nonhonors' students suggests that the scale could measure change 
equaily well for either group. Second, such differential change could be 
the result of statistical regression. 

Cook and Campbell (1979) argued that statistical regression to the 
pulation mean of a group may increase pre-post gain scores for low 
pretest scorers. Without benefit of a comparison control group, this 
interpretation cannot be ruled out. Including this variable in the path 
models, however, provided statistical control for estimating the effect 
of meeting the planning and developing careers competencies on voca- 
tional identity change. As stated earlier, VI pretest scores are unrelated 
to mastery of these guidance competencies. Pre-treatment differences 
in vocational identity did not have an indirect effect through these 
guidance competencies on posttreatment scores. Pretest scores remove 
that portion of vocational identity gain scores due to pretreatment 
differences. Thus, more confidence can be placed on th path between 
the planning and developing careers competencies and vocational iden- 
tity change because this relationship does not capitalize on such pre- 
treatment differences for either boys or girls. 

Cook and Campbell (" 979) identified several threats to interpreting 
one-group pretest-posttest designs. In this study, the absence of a control 
group necessitates ruling out several competing explanations to our 
ability to infer treatment-caused change. Possible effects due to history, 
maturation, testing, and statistical regression need to be considered. 
Using path analyses to explain vocational identity change, English 
grades, and meeting guidance competencies provides some statistical 
control where randomization and control group procedures could not be 
introduced into this field setting. Explaining observed changes due to 
'"istory and maturation seems untenable. Given the short duration of the 
tjmt and lack of identifiable intervening events capable of systematically 
influencing the junior class, such effects seem unlikely. If such events 

*e uncovered, however, their influence could be traced through the 



path models. Such is the case for possible effects due to testing And 
statistical regression. 

Cook and Campbell ( 1 979) argued that prior exposure to i;. , . r 
measure can cause posttest shifts in participant performance. As uz> 
scribed earlier, effects due to statistical regression could affect students' 
VI posttest scores. The path models, however, used VI pretest scores as 
a covariate explaining that portion of vocational identity change scores 
due to possible influences of prior testing and statistical regression. 
Thus, such effects can be ruled out as an explanation for the rsiauorship 
between guidance competency attainment and vocational ;d*r.i:ry 
change. Unfortunately, pre- post data on student self-repcrLs of rr, rating 
guidance competencies are not available. Future evaluation of the 
guidance and language arts unit will need to obtain these data. 

We reported results to counselors and teachers. From these iniiiai 
discussions, recommendations for unit improvement and a greater un- 
derstanding for how and why school counselors should engage in 
evaluation practices have begun to emerge. For example, counselors 
were concerned that only 54% of the girls indicated that they had 
explored careers different from those which they had already been 
considering prior to participating in the unit. This raised the question of 
how effective the guidance and language arts unit was in challenging 
emerging self-understandings that may restrict vocational aspirations 
within circumscribed occupational boundaries (Gottfredson, 1981; 
Lapan & Jingeleski, 1 992). Also, results highlighted particular problems 
displayed by boys in the nonhonors* group. Counselors believed that 
within this group the learning needs of many students at risk for schooi 
failure could be addressed. We hope such discussions will lead to a 
tailoring of the unit to serve the diversity of students within the junior,- 
class. The language arts faculty members are beginning to reexamine 
how students are introduced to the unit. 

Counselors indicated that certain professional development benefits 
accompanied their participation in evaluating the unit. For example, one 
counselor reported that gathering data on and assessing change in the 
tests used led to both a better understanding of some scales and different 
ways of interpreting results to students. Also, opportunities for coun- 
selor networking became more obvious. Packaging and sharing this' 
curriculum effort with counselors and school administrators in neigh- 
boring schools may be enhanced by the information and program 
refinement accompanying such evaluation efforts. Last, there may well 
be important motivational incentives inherent in this process. Asser- 
tively grappling with the effectiveness of existing guidance efforts may 
provide additional indicators that what a school counselor does makes 
a difference and a feeling of moving ahead to improve these efforts. The 
means need to be developed to promote this activity as a realistic part 
of a school counselor's job. 
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Appendix G 



Is the Guidance Program Having an Effect on the Climate/Goals of the 
District/Building? 

What judgments can be made about the effects of a guidance program on the 
chi .^ate/goals of the district/buildings? How do we conduct such an evaluation 
to make these judgments? Consider the following example. 

In the spring of 1989, the Ferguson-Florissant School District (Rain, Walker, 
& Jenkins, undated) initiated a program called Resolve All Problems Peacefully 
(RAPP) in Ferguson Middle School. It is a conflict-resolution program involving 
counselors, teachers, administrr* Jrs, parents, and students. It is an all-school 
program, part of the overall guidance program of that school. The program 
subsequently was initiated in other buildings and at other levels in the district. 

The primary purpose of the program was to decrease fighting in the school. 
Another purpose was to create in students a positive attitude toward others and 
toward the school. When these two goals were achieved a more pleasant and 
positive classroom atmosphere was expected to result. 

The program was put into operation by counselors who visited classrooms 
and explained the program. In addition, intensive training was provided to all 
who were to participate in the program — students, teachers, counselors, and 
administrators. The goal was to immerse the school fully in the goals and activities 
of the program. 

What were the results? Here is what happened at Ferguson Middle School: 

The conflict resolution program at Ferguson Middle School has been successful 
beyond anyone* s expectations. Mediators have helped their peers resolve their con- 
flicts peacefully about 100% of the time. There has rarely, if ever, been a fight between 
students who have reached a resolution in a RAPP mediation. Office referrals for 
the first quarter of the 1989-90 school year were down 75% aver the same time one 
year ago (before RAPP). Fights, which had been increasing every year, have dropped 
dramatically since the inception of RAPP. In the first semester of the 1986-87 school 
year, there were 66 fights, 1987-88, 73 fights, 1988-89, 88 fights, (RAPP began in 
the spring of '89), 1989-90, 36 fights. 

Many teachers have commented on the increased self esteem of RAPPers. These 
students are proud to be selected by their teachers and responsible for maintaining 
a peaceful school environment. We are noticing a change in the attitude of our 
student body. There is great prestige connected with being a member of RAPP 
and associating with lawyers and administrators. Many students want to be a 
part of RAPP and therefore model that behavior, It is now considered the ,4 in 
thing" to break up fights instead of starting them. (Ferguson Middle School, 
1990) 



Reprinted from Gysbers, N.C and Henderson P. (1994) Developing and Managing 
Your School Guidance Program (2nd Ed.) Alexandria, VA: American Counseling 
Association, pp 351-352. 
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MISSOURI COMPREHENSIVE GUIDANCE PROGRAM 
Student Survey 

Grade 9 10 11 12 



Sex M F 



What do you plan to do after high school? 

.Attend a vocational, technical, or business school. 

Attend a junior college. 

Attend a college or university. 

Get a job. 

Enter the military. 

Other (Please explain.) __ 

What type of classes have you taken or are you taking In high school? 

General 

Vocational-technical 

College Prep 

In which of the following activities have you participated with a 
counselor? (Check all that apply.) 

Individual sessions. 

Small groups. 

Classroom presentations. 

Workshops, seminars, or large groups. 

Other (Please explain. )_ 

Check the areas in which counselors or teachers have helped you during 
the year. (Check ail that apply.) 

Planning your high school classes. 

Planning and exploring careers. 

Planning for education and training after high school. 

Applying for admission to college, university, or vocational-technical 

schools. 

Finding financial aid information, 

Preparing to look for a job. 

- Understanding how being male or female relates to jobs and careers. 

Making decisions. 

Improving basic skills and study skills and doing better in classes. 

Understanding and getting along with others. 

Knowing how drugs and alcohol affect me and my friends. 

Understanding and accepting myself. 

Adjusting to high school. 

Interpreting test results. 

Learning about myself (my interests, abilities, etc.) 

Dealing with personal problems. 

Other (Please explain. )_ 

(Over) 
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Are the guidance program and the help the counselors can provide well 
publicized and known to students In your school? 

Yes ISb 

How did you learn about the guidance program and the work of the 
counselors in your school? 



In what ways have the guidance program and the counselors in your school 
helped you? 



What suggestions do you have for the guidance program in your school? 
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MISSOURI COMPREHENSIVE GUIDANCE PROGRAM 
Parent Survey 

Your Son's/Daughter's Grade 9 10 11 12 

Have you talked with your son's/daughters counselor this year? 
Yes No 



Please check the reason(s) for talking with your son's/daughter's 
counselor. 

Progress in school. 

Concerns about tardiness and attendance. 

High school course planning. 

College, trade school, and further education. 

Financial aid and admissions information. 

Relationships with teachers. 

Test results interpretation. 

Career exploration and planning. 

Personal, social, and/or emotional concerns. 

Parenting skills. 

Oth e r " 



How have you and your son/daughter benefited from the guidance program 
and the work of the counselor in your school? 



What suggestions do you have for the guidance program at your school? 
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MISSOURI COMPREHENSIVE GUIDANCE PROGRAM 
Teacher Survey 

Please rate the extent to which each of the following is being offered 
through the guidance program and the work of the counselors In your 
school- Use the following rating scale. 

1- A great deal 

2- Some 

3- Little 

4- Not at all 

5- Unaware of 

Guidance activities for students in classrooms on a regular basis. 

Individual and small group planning for students concerning their personal, 

educational, and career goals. 

Staff and parent consultation concerning student personal and academic progress. 

Individual counseling. 

Small group counseling. 

Student referral to other professionals and agencies as appropriate. 

Information to staff and community about the aims and purposes of the guidance 

program. 

Test score and other data interpretation for teachers about student performance. 



What impact has the guidance program and activities had on your students 
and school? 



What are the strengths of the guidance program in your school? 



What recommendations would you make for the guidance program in your 
school? 



ERLC 
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